给定数千种同样准确的机器学习（ML）模型，用户如何在其中选择？最近的ML技术使领域专家和数据科学家能够为稀疏决策树生成完整的Rashomon设置，这是一套几乎最理想的可解释的ML模型。为了帮助ML从业者识别具有此Rashomon集合中理想属性的模型，我们开发了Timbertrek，这是第一个交互式可视化系统，该系统总结了数千个稀疏决策树的规模。两种用法方案突出了Timbertrek如何使用户能够轻松探索，比较和策划与域知识和价值观保持一致的模型。我们的开源工具直接在用户的计算笔记本和Web浏览器中运行，从而降低了创建更负责任的ML模型的障碍。Timbertrek可在以下公共演示链接中获得：https：//poloclub.github.io/timbertrek。

机器学习（ML）可解释性技术可以揭示数据中的不良模式，这些模型模型开发以做出预测 - 一旦部署就会​​造成危害。但是，如何采取行动解决这些模式并不总是很清楚。在ML与人类计算机互动研究人员，医师和数据科学家之间的合作中，我们开发了GAM Changer，这是第一个互动系统，可帮助域专家和数据科学家轻松，负责任地编辑通用的添加剂模型（GAM）和修复有问题的模式。借助新颖的交互技术，我们的工具将可解释性置于行动中 - 使用户能够分析，验证和使模型行为与知识和价值相结合。医师已经开始使用我们的工具来调查和修复肺炎和败血症的风险预测模型，以及在不同领域工作的7位数据科学家的评估突出显示我们的工具易于使用，满足他们的模型编辑需求，并适合他们当前的工作流程。我们的工具以现代网络技术为基础，在用户的网络浏览器或计算笔记本电脑中本地运行，从而降低了使用的障碍。 GAM Changer可在以下公共演示链接中获得：https：//interpret.ml/gam-changer。

随着机器学习（ML）系统变得越来越普遍，有必要在部署之前审核这些系统的偏见。最近的研究开发了算法，以有效地以可解释的，表现不佳的数据（或切片）的形式有效地识别相互偏见。但是，这些解决方案及其见解是有限的，而没有用于视觉理解和与这些算法结果相互作用的工具。我们提出了Visual Auditor，这是一种交互式可视化工具，用于审核和汇总模型偏差。视觉审核员通过提供可解释的交叉偏差概述（检查由多个功能定义的人群，有问题的数据切片之间的关系以及在模型中表现不佳和表现表现不佳之间的比较之间存在的详细信息）来协助模型验证。我们的开源工具直接在计算笔记本和Web浏览器中运行，使模型审核可访问并易于集成到当前的ML开发工作流中。一项与Fiddler AI的域专家合作的观察用户研究强调，我们的工具可以帮助ML实践者识别和理解模型偏见。

语言模型既展示了定量的改进，又展示了新的定性功能，随着规模的增加。尽管它们具有潜在的变革性影响，但这些新能力的特征却很差。为了为未来的研究提供信息，为破坏性的新模型能力做准备，并改善社会有害的效果，至关重要的是，我们必须了解目前和近乎未来的能力和语言模型的局限性。为了应对这一挑战，我们介绍了超越模仿游戏基准（Big Bench）。 Big Bench目前由204个任务组成，由132家机构的442位作者贡献。任务主题是多样的，从语言学，儿童发展，数学，常识性推理，生物学，物理学，社会偏见，软件开发等等。 Big-Bench专注于被认为超出当前语言模型的功能的任务。我们评估了OpenAI的GPT型号，Google内部密集变压器体系结构和大型基础上的开关稀疏变压器的行为，跨越了数百万到数十亿个参数。此外，一个人类专家评估者团队执行了所有任务，以提供强大的基准。研究结果包括：模型性能和校准都随规模改善，但绝对的术语（以及与评估者的性能相比）；在模型类中的性能非常相似，尽管带有稀疏性。逐渐和预测的任务通常涉及大量知识或记忆成分，而在临界规模上表现出“突破性”行为的任务通常涉及多个步骤或组成部分或脆性指标；社交偏见通常会随着含糊不清的环境而随着规模而增加，但这可以通过提示来改善。

最近在可解释的机器学习中的进展（ML）研究表明，模型利用数据中的不良模式来进行预测，这可能导致部署危害。但是，尚不清楚我们如何解决这些模型。我们介绍了我们正在进行的工作，游戏改变者，一个开源交互式系统，以帮助数据科学家和领域专家轻松且负责任地编辑其广义添加剂模型（Gams）。通过新颖的可视化技术，我们的工具将可解释性投入到行动 - 使人类用户能够分析，验证和对齐模型行为与他们的知识和价值。使用现代Web技术建造，我们的工具在用户的计算笔记本或Web浏览器中在本地运行，而无需额外计算资源，降低屏障以创建更负责的ML模型。Gam更换器可在https://interpret.ml/gam-changer中获得。

数据增强是自然语言处理（NLP）模型的鲁棒性评估的重要组成部分，以及增强他们培训的数据的多样性。在本文中，我们呈现NL-Cogmenter，这是一种新的参与式Python的自然语言增强框架，它支持创建两个转换（对数据的修改）和过滤器（根据特定功能的数据拆分）。我们描述了框架和初始的117个变换和23个过滤器，用于各种自然语言任务。我们通过使用其几个转换来分析流行自然语言模型的鲁棒性来证明NL-Upmenter的功效。基础架构，Datacards和稳健性分析结果在NL-Augmenter存储库上公开可用（\ url {https://github.com/gem-benchmark/nl-augmenter}）。

A "heart attack" or myocardial infarction (MI), occurs when an artery supplying blood to the heart is abruptly occluded. The "gold standard" method for imaging MI is Cardiovascular Magnetic Resonance Imaging (MRI), with intravenously administered gadolinium-based contrast (late gadolinium enhancement). However, no "gold standard" fully automated method for the quantification of MI exists. In this work, we propose an end-to-end fully automatic system (MyI-Net) for the detection and quantification of MI in MRI images. This has the potential to reduce the uncertainty due to the technical variability across labs and inherent problems of the data and labels. Our system consists of four processing stages designed to maintain the flow of information across scales. First, features from raw MRI images are generated using feature extractors built on ResNet and MoblieNet architectures. This is followed by the Atrous Spatial Pyramid Pooling (ASPP) to produce spatial information at different scales to preserve more image context. High-level features from ASPP and initial low-level features are concatenated at the third stage and then passed to the fourth stage where spatial information is recovered via up-sampling to produce final image segmentation output into: i) background, ii) heart muscle, iii) blood and iv) scar areas. New models were compared with state-of-art models and manual quantification. Our models showed favorable performance in global segmentation and scar tissue detection relative to state-of-the-art work, including a four-fold better performance in matching scar pixels to contours produced by clinicians.

The number of international benchmarking competitions is steadily increasing in various fields of machine learning (ML) research and practice. So far, however, little is known about the common practice as well as bottlenecks faced by the community in tackling the research questions posed. To shed light on the status quo of algorithm development in the specific field of biomedical imaging analysis, we designed an international survey that was issued to all participants of challenges conducted in conjunction with the IEEE ISBI 2021 and MICCAI 2021 conferences (80 competitions in total). The survey covered participants' expertise and working environments, their chosen strategies, as well as algorithm characteristics. A median of 72% challenge participants took part in the survey. According to our results, knowledge exchange was the primary incentive (70%) for participation, while the reception of prize money played only a minor role (16%). While a median of 80 working hours was spent on method development, a large portion of participants stated that they did not have enough time for method development (32%). 25% perceived the infrastructure to be a bottleneck. Overall, 94% of all solutions were deep learning-based. Of these, 84% were based on standard architectures. 43% of the respondents reported that the data samples (e.g., images) were too large to be processed at once. This was most commonly addressed by patch-based training (69%), downsampling (37%), and solving 3D analysis tasks as a series of 2D tasks. K-fold cross-validation on the training set was performed by only 37% of the participants and only 50% of the participants performed ensembling based on multiple identical models (61%) or heterogeneous models (39%). 48% of the respondents applied postprocessing steps.

The ability to create realistic, animatable and relightable head avatars from casual video sequences would open up wide ranging applications in communication and entertainment. Current methods either build on explicit 3D morphable meshes (3DMM) or exploit neural implicit representations. The former are limited by fixed topology, while the latter are non-trivial to deform and inefficient to render. Furthermore, existing approaches entangle lighting in the color estimation, thus they are limited in re-rendering the avatar in new environments. In contrast, we propose PointAvatar, a deformable point-based representation that disentangles the source color into intrinsic albedo and normal-dependent shading. We demonstrate that PointAvatar bridges the gap between existing mesh- and implicit representations, combining high-quality geometry and appearance with topological flexibility, ease of deformation and rendering efficiency. We show that our method is able to generate animatable 3D avatars using monocular videos from multiple sources including hand-held smartphones, laptop webcams and internet videos, achieving state-of-the-art quality in challenging cases where previous methods fail, e.g., thin hair strands, while being significantly more efficient in training than competing methods.

Text-guided image editing can have a transformative impact in supporting creative applications. A key challenge is to generate edits that are faithful to input text prompts, while consistent with input images. We present Imagen Editor, a cascaded diffusion model built, by fine-tuning Imagen on text-guided image inpainting. Imagen Editor's edits are faithful to the text prompts, which is accomplished by using object detectors to propose inpainting masks during training. In addition, Imagen Editor captures fine details in the input image by conditioning the cascaded pipeline on the original high resolution image. To improve qualitative and quantitative evaluation, we introduce EditBench, a systematic benchmark for text-guided image inpainting. EditBench evaluates inpainting edits on natural and generated images exploring objects, attributes, and scenes. Through extensive human evaluation on EditBench, we find that object-masking during training leads to across-the-board improvements in text-image alignment -- such that Imagen Editor is preferred over DALL-E 2 and Stable Diffusion -- and, as a cohort, these models are better at object-rendering than text-rendering, and handle material/color/size attributes better than count/shape attributes.